Neural network architectures typically consist of massively parallel systems of simple computational elements. While software-based implementations are adequate for simulating these nonlinear dynamical systems, the physical realization of the true computational processing power inherent in such architectures can only be unleashed with their hardware implementation. This assumes that the electronic implementation retains the fine grained massive parallelism feature inherent in the model. There are a multitude of hardware approaches currently being taken for the implementation of neural network architectures, and these include: analog approaches; biologically motivated pulse-stream arithmetic approaches; optoelectronic approaches; charge coupled device approaches; and digital approaches.
The application of neural networks to problems that require adaptation (either from example or by self-organization based on the statistics of applied inputs) is among the most interesting uses of neural networks. In either case, a critical issue for any hardware implementation, is the inclusion of either on-chip or chip-in-the-loop learning capabilities based on one or more of the current learning paradigms. Real-time adaptation constraints might even further focus the on-chip learning requirements by specifying a need for the adjustment of the synaptic weights in a fully parallel and asynchronous fashion.
Of the numerous neuromorphic learning paradigms currently available, the broad majority are aimed at supervised learning applications. These range from simple Hebbian models with learning rules that require local connectivity information only, to complex hierarchical structures such as the Adaptive Resonance Theory (ART) model. Intermediate in complexity are algorithms for gradient descent learning that are most commonly applied to feedforward neural networks, and to a lesser extent to fully recurrent networks. These gradient descent algorithms are used to train networks from examples. Whether used for implementing a classification problem or a conformal mapping from one multidimensional space into another, adaptation involves selecting an appropriate set of input and output training vectors. Common to any supervised learning paradigm, training is achieved by applying an input to the network and calculating the error between the actual output and the desired target quantity. This error is used to modify the network weights in such a way that the actual output is driven toward the target. What differentiate models are the actual network topologies and the mathematical learning formalisms.